1. Field of the Invention
The present invention relates to a decoding/displaying device for decoding and displaying coded picture data which are generated by high-efficiency coding means using interframe correlation, and particularly to a device which can perform a continuous decoding operation and is suitable for use in implementing a function of converting 16:9 signals to letter-box signals, an audio data decoding function and an OSD (On Screen Display) function.
2. Description of Related Art
The data amount of picture data has recently increased more and more. In order to reduce the transmission or recording cost, these data are compressed by high-efficiency coding means for removing redundancy, etc., and then the coded picture data are transmitted or recorded. The high-efficiency coding means is briefly described in "MPEG Video Coding" Journal of the Television Society, vol. 49, No. 4 (1995), pp.435-466, by Nakajima et al., "Fundamentals of Digital Picture Compression" written by Yasuda and Watanabe, published by Nikkei BP Publication Center (Jan. 20, 1996), pp 135.varies.200, etc. The MPEG system which is standardized in ISO/SC29/WG11 has been well known. A decoding/displaying device for MPEG streams based on the MPEG system is disclosed in Japanese Patent Application Publication (Laid-open:KOKAI) No. Hei-8-18953 and No. Hei-8-23514.
In the coding based on the MPEG system, each frame of picture data is sectioned into an I picture (Intra Picture) which is coded without using a reference picture to be referred to as a prediction value, a P-picture (Predictive Picture) which is coded by using only pictures located at the forward side in the display order as reference pictures, and a B-picture (Bi-directional Picture) which is coded by using pictures located at the forward and rearward sides in the display order as reference pictures. In the actual coding operation, two reference pictures located at the forward and rearward sides must exist in the decoding operation of the B-picture, and the coding is performed after the pictures are skillfully rearranged (the order of the pictures is changed). Therefore, as shown in FIG. 15, it is necessary to decode the coded picture data (hereinafter a data stream containing coded picture data is referred to as "an MPEG stream") and then rearrange the data (i.e., arrange the data in a difference sequence) for display.
Therefore, in the decoding/displaying device, coded pictures which are transmitted in the coding order are successively decoded. However, the decoded data are temporarily stored in a memory and then rearranged so as to be in the display order. The decoded data of I and P pictures are required to be used as reference data for the subsequent decoding of B pictures, and thus it is necessary to store the picture data of two pictures in the memory. Further, in the case where one frame comprises two interlaced fields like television signals, even B pictures cannot be displayed simultaneously with the decoding operation thereof because one frame is coded as one picture. The frame data must be converted to field data, and thus the display operation must be delayed from the decoding operation by at least 0.5 frame period as shown in FIG. 15. Therefore, the decoded data are required to be temporarily stored in the memory to convert the frame data to the field data, and in order to satisfy this requirement, an extra memory area corresponding to one picture (=frame) is needed.
FIG. 14 is a diagram showing a memory which is required for the decoding of coded picture according to the MPEG system. The memory shown in FIG. 14 has a data width of 16 bits and a 16-Mbit memory capacity of 512 columns.times.2048 rows. One row corresponds to 1 Kbyte (=1024 bytes). The 16-Mbit capacity memory is used as a general memory chip in various fields, and the fabrication of a decoding/displaying device within the memory capacity range of 16 Mbytes is significant in cost.
The NTSC system of 720 horizontal pixels.times.480 vertical scan lines.times.the frame frequency of 30 Hz, and the PAL system of 720 horizontal pixels.times.576 vertical scan lines.times.the frame frequency of 25 Hz are used for picture signals to be compressed according to the MPEG system. FIG. 14 also shows a memory area sectioning style when the coded picture data corresponding to the PAL system having a large picture size are decoded. In the PAL system, 405 rows for brightness signals and 203 rows for color signals (the fractional portion is rounded up), a total of 608 rows, are needed for one frame, and the residual portion corresponding to three frames is used as a coded picture data buffer for temporarily storing the coded picture during the decoding operation. The capacity of the coded picture data buffer is equal to 1,835,008 bits (224 rows). This capacity value is the absolute minimum capacity which must always be maintained to ensure the optimum coding/decoding operation even in combination with a coding device and a decoding/displaying device which are manufactured by different makers, and it is determined according to the MPEG system.
FIG. 13 shows a conventional decoding/displaying device for coded picture data. In FIG. 13, reference numeral 1 represents a timing/operation mode control circuit, reference numeral 2 represents a parser/variable-length decoding circuit of MPEG streams, reference numeral 3 represents an inverse quantization/IDCT (Inverse Cosine Transform) circuit, reference numeral 4 represents a motion compensation circuit, reference numeral 5 represents a display circuit, reference numeral 6 represents a coded picture data write-in control circuit, reference numeral 7 represents a coded picture data read-out control circuit, reference numeral 8 represents a reference data readout control circuit, reference numeral 9 represents a decoded data write-in control circuit, reference numeral 10 represents a display picture data read-out control circuit, and reference numeral 11 represents a memory.
An MPEG stream is input to the parser/variable-length decoding circuit 2, and coded picture data are stored through the coded picture data write-in control circuit 6 into a coded picture data buffer area of the memory 11. The timing/operation mode control circuit 1 has not only a function of setting an operation mode, etc. for each block, but also a function of adjusting competition of data buses of the memory 11. The coded picture data read-out control circuit 7 reads out the coded picture data stored in the coded picture data buffer area in a stored order at a rate of approximately one picture (=frame) per frame period in synchronism with a synchronous signal of a display system.
The coded picture data which are read out from the memory 11 through the coded picture data read-out control circuit 7 are input into the parser/variable-length decoding circuit 2 again. The parser portion of the parser/variable-length decoding circuit 2 extracts system header information of the MPEG stream and coded mode information in a header portion of the coded picture data, and the information thus extracted is used in an internal variable-length decoding portion. In addition, the parser portion outputs the information to the timing/operation mode control circuit 1 to set the operation mode of each of the inverse quantization/IDCT circuit 3, the motion compensation circuit 4, and the display circuit 5. The variable-length decoding portion of the parser/variable-length decoding circuit 2 mainly decodes the coefficient data, etc. of the cosine transformation which have been subjected to the variable-length coding, and transmits the data thus decoded to the inverse quantization/IDCT circuit 3. Further, the inverse quantization/IDCT circuit 3 returns the coefficient data to a suitable scale in an inverse quantization portion, and converts the data to picture data in an IDCT portion.
The motion compensation circuit 4 reads out from the memory 11, through the reference data read-out control circuit 8, reference data (the decoded data of reference pictures) for motion compensation by using motion vector information in the coded mode information which is obtained by the parser/variable-length decoding circuit 2. The reference data are added to the picture data which are generated in the inverse quantization/IDCT circuit 3 to obtain decoded data. The decoded data are written into the memory 11 through the decoded data write-in control circuit 9. However, when the decoded data are the picture data of the I or P picture, one of the two reference picture areas of the memory in which older reference data are stored is updated. When the decoded data are the picture data of the B picture, the data are written into the B picture area.
The decoded data which are decoded and written into the memory 11 as described above are read out as display data by using the display data read-out control circuit 10, and transmitted to the display circuit 5. The display circuit 5 outputs the display data read out as decoded picture data from the memory 11 in synchronism with a synchronous display signal.
The operation of the conventional device will be described on the basis of the write-in and read-out operations of picture data of the above-described three pictures into the memory area (hereinafter referred to as "frame memory") with reference to FIG. 17.
In the MPEG system, there have been known a coding case in which the coding is performed in the order of I1, B2, B3, P4, B5, B6, P7, . . . (numeral represents the display order) and a coding case in which the coding is performed with only I and P frames in the order of I1, P2, P3, P4, . . . However, an irregular arrangement pattern which does not conform to the regular order as described above is also permitted. FIG. 7 shows such an irregular arrangement pattern.
As shown in (e) of FIG. 17, each frame is assumed to be coded in the display order of I1, B2, B3, P4, P5, P6, . . . Further, in FIG. 17, the decoding operation of writing the decoded picture data into each frame memory (hereinafter referred to as FM1, FM2, FM3) is shown as being performed twice.
Each frame memory FM1, FM2, FM3 is illustrated as being divided into memory images of two field memories M1, M2 at the upper and lower sides. The upper half M1 corresponds to a field memory for a first field, and the lower half M2 corresponds to a field memory for a second field. A memory image in which the address is increased in the raster scanning order from the upper side to the lower side in the direction of arrows is shown for each of the field memories M1, M2.
The frame memories FM1, FM2 are used as frame memories for reference pictures, and the frame memory FM3 is used as a frame memory for B pictures. The arrows which are directed downwardly from (a) of FIG. 17 to (b), (c) or (d) of FIG. 17 indicate decoding and writing operations, the arrows which are directed upwardly from (b), (c) or (d) of FIG. 17 to (a) of FIG. 17 indicate a reference read-out operation, and the arrows which are directed downwardly from (b), (c) or (d) of FIG. 17 to (e) of FIG. 17 indicate a display read-out operation.
In the prior art, the decoding operation is progressed on a macro block basis according to a fixed time slot, and three types of memory accesses occur during the progress of the decoding operation. The decoding and writing operation is performed to write decoded picture data which are decoded on a macro block basis. The writing progress in the decoding and writing operation is not continuous, but discontinuity of addresses occurs during the progress. However, in the decoding and writing operation over one frame, the writing address is gradually increased. In (b), (c) and (d) of FIG. 17, dense crosshatched lines each having a broad width (for example, line 1 in (b) of FIG. 17) indicates the writing progress in the decoding and writing operation.
The reference read-out operation is performed to read out the decoded picture data of a reference frame as reference picture data on a macro block basis when the P frame and the B frame are decoded. The reading progress in the reference read-out operation is the same as the decoding and writing operation. In (b) and (c) of FIG. 17, thin crosshatched lines each having a broad width (for example, line 2 in (b) of FIG. 17) indicate the reading progress in the reference reading operation. A positive or negative offset value is added to the read-out address in accordance with the value of the motion vector indicating the shift amount of the macro block, and thus the width is generally broader than in the case of the decoding and writing operation.
The display read-out operation is performed to read out the decoded picture data which are held in the frame memory. The reading progress in the display read-out operation is continuous. However, the display read-out operation is temporarily stopped during a vertical blanking period between first and second fields. In (b), (c) and (d) of FIG. 17, a bold solid line (for example, line (3) in (b) of FIG. 17) indicates the reading progress of the display and read-out operation.
First, in a period T1 (one frame period), when an I1 frame is decoded, the respective data of the two fields of the decoded I1 frame are simultaneously written into the different field memories M1 and M2 of the frame memory FM1 as shown in (b) of FIG. 17 (decoding and writing operation 1).
In a next period T2 (one frame period), a P4 frame is decoded. In this case, as shown in (b) of FIG. 17, the II frame which is decoded from the frame memory FM1 is read out (reference read-out operation 2), and the P4 frame is decoded by using the read-out I1 frame as a reference frame. As shown in (c) of FIG. 17, the respective data of two fields of the decoded P4 frame are simultaneously written into the different field memories M1, M2 of the frame memory FM2.
In the last one field period of the period T2, as shown in (b) and (e) of FIG. 17, the first field of the I1 frame decoded from the upper half field memory M1 of the frame memory FM1 is read out (display read-out operation 3), and set as the first field of the I1 frame in the decoded picture data to be displayed.
In a next period T3 (one frame period), the B2 frame is decoded. In this case, as shown in (b) and (c) of FIG. 17, the I1 frame decoded from the frame memory FM1 and the P4 frame decoded from the frame memory FM2 are respectively read out, and the B2 frame is decoded by using the I1 frame and the P4 frame thus read-out as reference frames. The data of two fields of the decoded B2 frame are simultaneously written into the different field memories M1 and M2 of the frame memory FM3 as shown in (d) of FIG. 17. In this period T3, as shown in (b) and (e) of FIG. 17, the second field of the I1 frame which is decoded from the lower half field memory M2 of the frame memory FM1 are read out during the first half of period T3 (first field period), and as shown in (d) and (e) of FIG. 17, the first field of the B2 frame which is decoded from the upper half field memory M1 of the frame memory FM3 is read out during the last half of period T3 (second field period). These fields thus read out are set as the second field of the I1 frame and the first field of the B2 frame respectively in the decoded picture data to be displayed.
In a next period T4 (one frame period), the B3 frame is likewise decoded, and written into the frame memory FM3 as shown in (d) of FIG. 17. At this time, the second field of the B2 frame decoded from the lower half field memory M2 of the frame memory FM3 is read out in the first half of the period T4 (first field period), and the first field of the B3 frame decoded from the upper field memory M1 of the frame memory FM3 is read out in the last half of period T4 (second field period). These fields are set as the second field of the B2 frame and the first field of the B3 frame respectively in the decoded picture data to be displayed.
In a next period T5 (one frame period), the P5 frame is decoded. In this case, as shown in (c) of FIG. 17, the P4 frame decoded from the frame memory FM2 is read out, and the PS frame is decoded by using the P4 frame as a reference frame. The data of the two fields of the decoded P5 frame are simultaneously written into the field memories M1, M2 of the frame memory FM1 as show in (b) of FIG. 17. In the first half of the period T5, the second field of the B3 frame decoded from the lower half field memory M2 of the frame memory FM3 is read out as shown in (d) and (e) of FIG. 17, and in the last half of the period T5, the first field of the P4 frame decoded from the upper half field memory M1 of the frame memory FM2 is read out as shown in (c) and (e) of FIG. 17. These fields are set as the second field of the B3 frame and the first field of the P4 frame respectively in the decoded picture data to be displayed.
In a next period T6 (one frame period), the P6 frame is decoded. The P5 frame written into the frame memory FM1 is used as a reference frame for the P6 frame. In this case, the P5 and P6 frames correspond to the I1 and P4 frames respectively, and the same decoding processing as the I1 and P4 are performed on the P5 and P6 frames. Further, the decoding of an I or P frame subsequent to the above frames is performed by using the decoded P6 frame as a reference frame.
As described above, the frame decoding order as shown in (a) of FIG. 17 is converted to a frame display order which is suitable for the display as shown in (e) of FIG. 17.
FIG. 16 relates to the case where the B frames are sequential in the decoding/displaying processing, and shows the motion of the memory address of the frame memory FM3 for B frames. In FIG. 16, the memory address increases upwardly with the time lapse as indicated by arrows (this is just inverse to FIG. 17). The most minute grid on the ordinate axis in FIG. 16 corresponds to the block size for coding (8 lines in field). In FIG. 16, the number of blocks in one frame is not accurately illustrated. As shown in FIG. 16, the read-out of the display data is performed by reading out the data while successively incrementing the addresses of areas which are exclusively allocated to the respective fields in correspondence to the display period of each field. On the other hand, the write-in operation of the decoded data is performed over two fields and with an address width corresponding to the width of a block as indicated by a rectangle, because the coding is performed on a two-dimensional block basis after two fields are integrated into one picture (=frame). Further, the decoding operation is stopped (suspended) for a period containing a vertical blanking period at the switch point between the pictures in order to avoid competition between the read-out address and the write-in address in which the read-out address outpaces the write-in address.